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ABSTRACT 

A topic of current practical interest is the accurate characterization of the synoptic-scale atmospheric state 
from wind profiler and radiosonde network observations. We have examined several related and commonly 
applied objective analysis techniques for performing this characterization and considered their associated level 
of uncertainty both from a theoretical and a practical standpoint. A case study is presented where two wind 
profiler triangles with nearly identical centroids and no common vertices produced strikingly different results 
during a 43-h period. We conclude that the uncertainty in objectively analyzed quantities can easily be as large 
as the expected synoptic-scale signal. In order to quantify the statistical precision of the algorithms, we conducted 
a realistic observing system simulation experiment using output from a mesoscale model. A simple parameter- 
ization for estimating the uncertainty in horizontal gradient quantities in terms of known errors in the objectively 
analyzed wind components and temperature is developed from these results. 


1. Introduction 

The installation of the NOAA wind profiler dem- 
onstration network (WPDN) has significantly ex- 
panded opportunities for applying spatial objective 
analysis (SOA) techniques to data for diagnostic ap- 
plications, particularly in the Kansas-Oklahoma re- 
gion where a hexagonal network of profilers, spaced at 
approximately 175 km (Fig. 1), has been operational 
since mid 1991. These profilers operate at a frequency 
of 404.37 MHz and provide horizontal wind observa- 
tions in 250-m vertical increments from 500 m above 
ground level to approximately 16 km with nominal 1- 
h time resolution. The potential for WPDN data to char- 
acterize the meso-synoptic-scale state of the atmo- 
sphere has attracted several field programs to the mid- 
western United States. Among these are the second 
field phase of the First ISCCP (International Satellite 
Cloud Climatology Project) Regional Experiment 
(FIRE Cirrus H; NASA 1991) and the Atmospheric 
Radiation Measurement (ARM) Program (Stokes and 
Schwartz 1994). ARM has augmented the WPDN in- 
ner profilers with five radiosonde sites (Fig. 1) that 
collect soundings at high temporal resolution during 
frequent campaigns. 

Integrating major field efforts as closely as possible 
with a dense spatial network of wind profilers seems 
an obvious choice. However, the scientific rationale be- 
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hind this choice deserves comment. In the case of FTRE 
Cirrus II, the choice was motivated by the possibility 
of diagnosing meso- and synoptic-scale vertical mo- 
tions from divergence profiles deduced from profiler 
triangles. ARM seeks to extend this idea by attempting 
to diagnose the synoptic-scale advective tendencies of 
momentum and temperature from data for the purpose 
of integrating a single-column general circulation 
model (SCM). The SCM concept (Stokes and 
Schwartz 1994) relies on highly accurate diagnoses of 
the GCM-resolvable atmospheric state (200-km grid 
spacing and 1-h temporal resolution). The implied as- 
sumption is that the temporal history of five radiosonde 
profiles and seven wind profiles separated by approxi- 
mately 200 km can accurately characterize the advec- 
tive tendencies of the synoptic-scale atmosphere. In this 
paper we critically examine this assumption. In partic- 
ular, we investigate the magnitude of objective analysis 
uncertainties and their effect on synoptic-scale diag- 
nostic quantities valid at the center of the inner array 
of the WPDN. 

A considerable amount of work has been published 
recently on spatial objective analysis of wind profiler 
and radiosonde data for the purpose of diagnostic anal- 
ysis. Thiebaux and Pedder (1987) provide an excellent 
overall treatment of the subject, while recent articles 
by Davies-Jones (1993), Michaels (1994), and Za- 
mora et al. ( 1994) build on Thiebaux and Pedder’ s ef- 
forts. These works all consider the theoretical uncer- 
tainty of SOA techniques applied to data. The theo- 
retical uncertainties, however, are typically quantified 
in terms of parameters not known a priori, that is, the 
scale length of dominant meteorological features or the 
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Fig. 1 . Map of the inner array of the wind profiler demonstration 
array and the Atmospheric Radiation Measurement Program’s 
(ARM) southern Great Plains (SGP) site. The filled squares mark the 
location of wind profilers. Diamonds show the location of special 
radiosonde launch facilities installed by ARM and the circle marks 
the location of the ARM central facility. The dashed rectangle shows 
the conceptual GCM grid box used by ARM in single-column mod- 
eling applications. Profiler locations are Haviland, Kansas (HVL), 
Hillsboro, Kansas (HBR), Neodesha, Kansas (NDS), Vici, Oklahoma 
(VCI), Purcell, Oklahoma (PRC), Haskell, Oklahoma (HKL), and 
Lamont, Oklahoma (LMN). 


error in the observations. What is needed from the 
standpoint of the quasi-operational SCM approach en- 
visioned by ARM is some practical quantification of 
objective analysis precision in terms of observable pa- 
rameters, as well as a technique that will identify con- 
ditions where the uncertainties in the analysis products 
become large enough to negate further quantitative 
consideration. We address these issues in this paper. In 
the following section we briefly review SOA method- 
ology and discuss theoretical limitations. To place the 
theoretical discussion into a more practical context, a 
case study is presented to illustrate the influence of ob- 
jective analysis error on the diagnosis of horizontal di- 
vergence. In section three, we address the practical lim- 
itations of diagnosing field characteristics from small 
data networks using statistics derived from an observ- 
ing system simulation experiment. Based on these sta- 
tistics, a simple parameterization of objective analysis 
uncertainty is developed based on observable parame- 
ters. 

2. Methodology, uncertainty, and a practical 

illustration 

In order to place the following discussion in a rig- 
orous context, it is helpful at this point to briefly review 


some of the applicable theoretical development found 
in Thiebaux and Pedder (1987) and Davies-Jones 
(1993). An observation X(x, y, z, t) is assumed to 
have two basic components: /x(jc, y, z, t) represents the 
portion of the observation containing the desired at- 
mospheric signal and e(x, y, z, t) accounts for the por- 
tion of the observation composed of observational error 
and atmospheric signal of unwanted scale. It is assumed 
further that c is uncorrelated with /i in the spatial and 
temporal domains. The goal of the objective analysis 
technique is to minimize the influence of e and to en- 
sure the diagnosed characteristics are derived, as much 
as possible, from Following Thiebaux and Pedder 
(1987), a signal-to-noise ratio y is defined as the ratio 
of the mean-square deviations of the desired atmo- 
spheric signal to the mean square deviations of the 
noise, 

7 <U-<e»V 

where the angle brackets denote the averaging operator 
applied to some number of spatially distributed obser- 
vations. Owing to its assumed stochastic nature, (e) = 0 
for some appropriate ensemble average. Therefore, 

(Se - (m» 2 > 

r <(0 2 > ' 

Since fx is never known a priori, maximizing y reduces 
to the problem of minimizing the sum of square resid- 
uals, ( g - X 0 ) 2 , where g(x°, y°, z°, t°) is the analysis 
result at an observing location and X° represents the 
spatially distributed observations. 

As shown by Thiebaux and Pedder (1987) and Da- 
vies-Jones (1993), all practical applications of spatial 
objective analysis, whether stated explicitly or not, 
make a determination as to the magnitude and impor- 
tance of £ in a set of observations. For example, in the 
general technique described here, the field character- 
istics of the meteorological quantity are assumed to 
conform to some predetermined functional model. The 
characteristics of this model are imposed on the obser- 
vations, leading to a spatially continuous mathematical 
approximation of the field. A straightforward example 
of this procedure is exact fitting of planar surfaces to 
data from station triangles. This includes Bellamy's 
(1949) graphical technique, line integral methods 
(Endlich and Clark 1963), and the linear vector point 
function (Zamora et al. 1987 ) . Since the planar surface 
is fitted exactly to a minimal set of observations, no 
separation of signal from noise is possible. The implied 
assumption is that e is of negligible consequence to the 
analysis result. 

An immediate extension of the exact linear tech- 
niques is a generalized procedure that uses an arbitrary 
functional surface as the approximating model (in this 
work, we will consider polynomial surfaces). Thiebaux 
and Pedder show that this problem reduces to solving 
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[(F°) r F°] 0 = (F") 1 *" (1) 

for 0, provided that [(F°) T F°]~ l exists. In Eq. (1), a 
spatially distributed set of observations is contained in 
the vector X, while the specified functional form of the 
assumed data field is contained in the so-called design 
matrix F. The desired structural characteristics of the 
field variable (network mean and spatial derivatives) 
are returned in the vector 0 . When the number of ob- 
servations exactly matches the number of variables re- 
quired to define the functional surface, that surface is 
fitted exactly. However, when more observations are 
available than are needed to define the surface, the sur- 
face is fitted to the observations in a least squares sense 
by minimizing the sum of squared residuals between 
the analysis results and the observations. It can be 
shown easily that overdetermined least squares solu- 
tions effectively maximize the signal to noise ratio de- 
fined above. 

The issue we seek to address is the degree of preci- 
sion that can be ascribed to the diagnosed structural 
characteristics of the data field. Uncertainties in 0 arise 
from observational noise, e (contained in the vector 
X ), and from the degree to which the actual field struc- 
ture departs from the functional specification defined 
in the matrix F (also known as truncation error). As- 
suming that errors in the observations follow their theo- 
retical estimates, the former source of uncertainty can 
be estimated (Davies-Jones 1993), and this estimate 
should always be considered as a lower limit to the 
uncertainty in 0. The influence of c in a set of obser- 
vations tends to be minimized by spatially extensive 
and overdetermined data arrays. To evaluate the influ- 
ence of truncation error, however, the actual spatial 
characteristics of the ambient field must be known. 
Since determining these spatial characteristics is typi- 
cally the very purpose of the objective analysis exer- 
cise, the requisite knowledge needed to evaluate this 
error is generally unavailable, making truncation error 
an extremely troublesome source of uncertainty. Unlike 
observational error, truncation error tends to be mini- 
mized by decreasing the spatial extent of the data array 
used in Eq. (1), thus effectively sampling the feature 
under consideration in a more continuous fashion. For 
a fixed network of stations, this equates to decreasing 
the overdetermination of the functional surface and 
guarantees an increasing uncertainty in 6 due to obser- 
vational noise. 

Consideration of the objective analysis uncertainty 
and its component sources must be predicated on a 
careful assessment of the overall analysis goals dictated 
by 1 ) the chosen scale of signal under investigation and 
2) the degree of precision required in the diagnosed 
results. From a practical standpoint, it is generally im- 
possible to separate uncertainty arising from observa- 
tional noise and that due to truncation error in any given 
solution of Eq. ( 1 ) applied to data. With the exception 
of obviously erroneous observations, the observational 


noise component c in a particular observation is both 
unknown and often not wholly uncorrelated from c in 
observations from other nearby instruments as was as- 
sumed in the development of Eq. ( 1 ) . Beyond this, the 
magnitude of truncation error can, at best, only be es- 
timated crudely. The objective analysis exercise, then, 
becomes one of minimizing the resultant uncertainty 
arising from these oppositely trending sources of error 
and recognizing when the total uncertainty becomes so 
large, relative to the stated analysis goals, to negate 
further quantitative consideration of the results. 

In our particular application (diagnosing synoptic- 
scale quantities for a single-column GCM), we adopt 
the ARM convention and consider the entire inner array 
of the WPDN as approximating a single-grid cell. We 
then examine planar polynomial approximations to the 
data fields that use between three and seven observa- 
tions. The observing site configurations are approxi- 
mately centered on the inner array centroid near La- 
mont, Oklahoma. For brevity we denote the particular 
objective analysis model by the number of observations 
used. Exact linear solutions using station triangles, for 
example, are termed L3, while an overdetenmined lin- 
ear solution to data from all seven inner array profilers 
is denoted by L7. The atmospheric signal we wish to 
resolve has dominant horizontal scales greater than 
1000 km and we assume, following Holton (1979), 
that the quantities of interest scale as listed in Table 1. 
Establishing an accuracy standard for the diagnosed 
quantities is a somewhat ambiguous task that reduces 
to determining how much error is acceptable in any 
given realization of each quantity. Clearly the diag- 
nosed quantities must be more accurate than the range 
over which that quantity scales. Reasoning that the in- 
ner array of the WPDN spans approximately one-quar- 
ter of a typical synoptic wave, we postulate that an 
objective analysis uncertainty less than 25% of the 
scale value implies a well-resolved quantity, while an 
uncertainty between 25% and 50% of the scale value 
implies a marginally resolved quantity. Uncertainties 
in excess of 50% of the scale value is considered poorly 
resolved. 

A lower limit to the uncertainty in the diagnostic 
quantities can be estimated using the accepted rms un- 
certainty in the observations and the techniques de- 
scribed by Thiebaux and Pedder (1987) and Davies- 


Table 1. Typical magnitudes of synoptic-scale quantities. Div and 
Vor denote horizontal divergence and relative vorticity, 
respectively. 


W, V 

10 m s~ l 

T 

5 K 

diddx , Qu!dy % dvldx , dv/dy 

1 x 10~ 5 s' 1 

dT/dx, dTIdy 

0.5 x lO' 5 K m' 1 

Div. Vor 

1.4 x 10" 5 s' 1 

(V V, U ), (v-V,v) 

10 x 10~ 5 m s~ ! 

(V-V*D 

5 x lO' 5 K s' 1 


V 


..C'X''! mo Mnr (WOO PM AMC- MWP Mnnp QM 1HQ 


004 


MONTHLY WEATHER REVIEW 


Volume 000 


Table 2. Uncertainty in objectively analyzed quantities at the 
center of the WPDN inner array due only to rms observational error. 
The observing sites used are: L3-HVL, NDS, PRC; L4-VC1, PRC, 
HKL, HBR; L5-HVL, HBR, NDS. PRC. VC1; L6-HVL. NDS. 
HBR, PRC, VC1, HKL (see Fig. 1 for profiler locations). An rms 
observational error of 1.5 m is assumed for the horizontal wind 
components and 0.5 K for the temperature. For the advective 
acceleration, the error corresponds to the individual components of 
the vector quantity. 




L3 

L4 

L5 

L6 

w, v (m s“‘) 


0.90 

0.76 

0.71 

0.62 

T (K) 


0.25 

0.26 

0.24 

0.21 

S( “’ u) ao~ 3 s -l > 


0.69 

0.68 

0.59 

0.48 

dx 

*“• tier’s-') 


0.65 

0.54 

0.51 

0.48 

Oy 

Li(10' 5 Km- 1 ) 
ox 


0.23 

0.23 

0.20 

0.16 

£(.0- 3 K m-’) 
oy 

Div. Vor. (10 3 s' 


0.21 

0.17 

0.17 

0.16 


0.95 

0.87 

0.78 

0.68 

Adv. accel. (10 -3 

m s“ 2 ) 

7.8 

7.1 

6.4 

5.6 

Temp. adv. (10“ 3 

Ks-') 

4.5 

4.2 

3.8 

3.3 


Jones (1993). Assuming the hourly horizontal wind 
components are known to within about 1.5 ms" 1 
(Strauch et al. 1987) and radiosonde-observed temper- 
atures are known to within 0.5 K (Belt and Fuelberg 
1982), the theoretical uncertainties in the diagnostic 
quantities of interest (those listed in Table 1 ) are shown 
in Table 2. Not surprisingly, the wind components and 
temperature are well resolved for all the objective anal- 
ysis models considered. These quantities can be inter- 
polated to the array centroid with an accuracy less than 
or equal to the assumed observational uncertainty. 
However, the first-order spatial derivatives and terms 
derived from them do not fare so well. In general, quan- 
tities derived from both station triangles and the four- 
station network just marginally resolve synoptic-scale 
variability. While the situation improves somewhat for 
data networks of from five to seven observing sites, 
none of the quantities that rely on the first-order spatial 
derivatives are well resolved. Given that the level of 
precision listed in Table 2 is the maximum that can be 
expected from objectively analyzed data, any addi- 
tional uncertainty brought about by larger than antici- 
pated observational error or from any truncation error 
is indeed a serious concern. 

To illustrate this latter point, we present a case study 
designed to highlight the influence of nonlinearities in 
the wind field and their influence on diagnosed hori- 
zontal divergence. We consider two profiler triangles, 
the first composed of the HVL, NDS, PRC profilers 
and the second composed of the VCI, HKL, HBR pro- 
filers. We denote the triangles as T1 and T2, respec- 
tively. For comparison purposes, we also examine re- 
sults from a quadratic surface fit to data from a station 
array composed of the LMN, VCI, HVL, HBR, NDS, 


and PRC profilers (see Fig. 1 ). Note that each of the 
triangles is very nearly equilateral, while the triangles 
share no common vertices and have nearly identical 
centroids. This particular profiler geometry allows us 
to examine the influence of truncation error on the wind 
field diagnostics. As long as the wind field varies 
smoothly over the inner array, planar and higher-order 
polynomial surfaces fit to data will return identical 
wind field diagnostics within the bounds of observa- 
tional uncertainty. As the wind field increases in com- 
plexity relative to the geographic region bounded by 
the inner array, the wind field diagnostics derived from 
the data will diverge. 

Wind profiler data from 1500 UTC 25 November 
1991 (1500/25) to 0900 UTC 27 November 1991 
(0900/27) are examined. During the 43-h period, all 
inner array profilers were operational and generated 
data of reasonable quality. In order to simplify inter- 
pretation and suppress observational error in the hourly 
winds, data were smoothed with a low-pass filter (Kay- 
lor 1977) to remove oscillations with a period of 6 h, 
and less and missing values were replaced using an 
interpolation scheme described by Akima (1978) and 
Akima (1984). Less than 10% of the data were miss- 
ing, however. The evolution of the wind field during 
the period is shown by the time-height section of wind 
profiles from the LMN profiler (Fig. 2). The salient 
features of this period include the jet streak entrance 
region observed at Lamont early in the case study and 
the associated trough passage at 0000/26. Winds in the 
lowest layers remained west to southwesterly through- 
out this period. After 0300/26, winds became westerly 
and decreased in speed through much of the tropo- 
sphere in response to a nearly stationary trough and 



Fig. 2. Time-height cross section of horizontal winds observed by 
the Lamont, Oklahoma, profiler (site LMN in Fig. 1) from 1500 UTC 
25 November 1991 to 0900 UTC 27 November 1991. Time runs from 
right to left in the diagram, contours are in meters per second, and 
vectors are compass direction (north being towards the top of the 
page) toward which the winds are blowing. 
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Fig. 3. Scaner plots of the horizontal velocity gradient components determined by exact linear fits (L3 
objective analysis model) to triangles composed of the HVL, NDS, and PRC profilers (triangle Tl, abscissa) 
and the VCI, HKL, and HBR profilers (triangle T2, ordinate). Units are 10" 5 s~ l : (a) du/dx, (b) du/dy, (c) 
dv/dx , (d) dv/dy. 


diffluent flow pattern that was established over the cen- 
tral United States. After 1800/26, the winds backed 
further and an abrupt local acceleration occurred as the 
wind veered after 2300/26. 

Figure 3 shows the correlation of the first-order spa- 
tial derivatives at all vertical levels estimated by the 
exact linear method applied to the two triangles. Some 
degree of linear correspondence exists in the scatter- 


Table 3. The uncertainty in the indicated objectively analyzed 
quantities derived from the wind profiler polygons used in the case 
study. An rms observational error of 1.5 m s' 1 is assumed and units 
are 10“ 3 s"‘. 



d(u, v) 
dx 

d(u, u) 

dy 

Divergence, 

Vonicity 

HVL, NDS, PRC 

0.69 

0.65 

0.95 

VCI, HKL, HBR 

0.87 

0.74 

1.1 

LMN, VCI, HVL, 
HBR, NDS, PRC 

1.3 

1.1 

1.7 


plots and no obvious biases appear to exist in the data. 
However, significantly more scatter is evident in these 
plots than would be expected by considering only the 
influence of observational uncertainty on the analysis 
results (Table 3). Also, the scatterplot of dv/dy sug- 
gests that a number of the diagnoses of this term display 
marked negative correlation. 

Time series plots of the horizontal divergence and 
the component terms at two selected levels (Figs. 4 and 
5) show that the degree to which the linear estimations 
differ is related to meteorological features that passed 
over the inner array. The du/dx term at 1.5 km shows 
close agreement to within the observational uncertainty 
through most of the period. The only significant dis- 
agreement between the linear estimates occurs near 
0300/26. A period of large uncertainty is also noted in 
the dv/dy term between 0900/26 and 1800/26. Note 
that the horizontal divergence estimated using the 
quadratic fit at 1.5 km closely agrees with the linear 
estimation from triangle Tl. This agreement is, how- 
ever, fortuitous and due to a cancellation of differences 
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27 Nov 26 Nov 25 Nov 

Fig. 4. Time series of (a) duJdx , (b) &v(dy t and (c) horizontal di- 
vergence ( du/dx + dvldy) calculated at 1.5 km using exact linear 
polynomial fits (L3 objective analysis model) to wind profiler trian- 
gles composed of the HVL, NDS, and PRC profilers (triangle Tl, 
solid circle) and the VCI, HKL, and HBR profilers (triangle T2, open 
circle). The curve denoted by solid triangles uses an exact quadratic 
fit (Q6 in the legend) to profiler data from the HVL. HBR, NDS, 
PRC, VCI, and LMN profilers. The abscissa has units of I0 -3 s~ J . 
Time runs from right to left as in Fig. 2 from 1400 UTC 25 November 
1991 to 1000 UTC 27 November 1991. 


in the individual terms as can be noted by examining 
Figs. 4a and 4b. A more accurate depiction of the un- 
certainty in the horizontal divergence is seen by com- 
paring the two linear estimates where an average dif- 
ference of 6 X 10 ~ 5 s“* occurs during this period. The 
magnitude of this difference is alarming since it is 
nearly as large as the expected range of atmospheric 
signal over an entire synoptic-scale system and more 
than an order of magnitude larger than the minimum 
uncertainty estimated using only the assumed rms error 
in the observations. 

Large differences in the two linear estimates in the 
upper troposphere (Fig. 5) are evident throughout the 
case study. Agreement to within the observational un- 
certainty occurs in only 8 of the 43 hours under con- 
sideration. The linear solutions tend be negatively cor- 
related much of the time and the quadratic solution 
tends to remain midway between them. This is most 
vividly displayed by the dvldy terms between 1800/26 
and 0400/27. During this time a trough and jet streak 
exit region influenced the Oklahoma and Kansas region 
(Mace et al. 1995). It should be noted that this exit 
region was part of a synoptic-scale feature that passed 
rapidly over the middle United States and generated a 
propagating band of mid- and upper-level cloudiness. 


In spatial objective analysis, the values of the diag- 
nosed horizontal derivatives usually are considered 
valid at the observational centroids of the polygons. In 
this case, the centroids of the two triangles and the six- 
station array are nearly identical, but the wind field 
characteristics derived from them are markedly differ- 
ent. This is a vivid example of the difficulties that must 
be considered when objective analysis results are to be 
used for quantitative applications such as calculation of 
vertical motions or construction of boundary conditions 
for single-column models. It is these types of strongly 
forced situations that must be accurately resolved if 
data-derived diagnostics are to be used successfully. 
For instance, the SCM concept adopted by ARM as- 
sumes that a meteorologically diverse and statistically 
significant set of well-characterized cases will compose 
the database against which cloud and radiation param- 
eterizations will be tested (Stokes and Schwartz 1994). 
Periods contaminated by objective analysis error must 
be filtered from this database. Given the marginal pre- 
cision of inner-array data polygons to resolve synoptic- 
scale signal, identification of poorly resolved cases is 
crucial if objectively analyzed wind profiler and radio- 
sonde network data are going to be credible sources of 
information. 

As demonstrated by the case study, the uncertainty 
in derivative estimates can easily be an order of mag- 
nitude larger than the uncertainty calculated by consid- 
ering only the assumed uncertainty in the observations. 
This additional uncertainty arises due to a combination 
of truncation error and a spatial and temporal depen- 



27 Nov 26 Nov 25 Nov 

Fig. 5. As in Fig. 4 except at 9.5 km. 
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dence of observational error not accounted for in the 
development of Eq. ( 1 ). The point we wish to stress is 
that neither of these additional sources of uncertainty 
can be known a priori nor can they be evaluated rig- 
orously in a postanalysis mode. Furthermore, it is gen- 
erally impossible to evaluate separately the sources of 
objective analysis error in any given application of Eq. 
( 1 ) to data. While the work of previous authors has 
been necessary to highlight theoretical limitations (Da- 
vies-Jones 1993; Zamora et al. 1994; Michaels 1994; 
Doswell and Caracena 1987), what is needed from a 
practical standpoint is some quantification of the pre- 
cision that can be expected from applying objective 
analysis algorithms to data when a realistic convolution 
of both observational error and truncation error are pos- 
sible. A straightforward technique is required that will 
aid the analyst in identifying situations where the un- 
certainty in derivative estimation is likely to be so large 
as to negate further quantitative consideration of a par- 
ticular meteorological case. 

3. Estimates of precision and a technique for error 

discrimination 

In order to examine the precision of diagnostic quan- 
tities derived from data using the algorithms described 
in the previous section, some objective measure of the 
true atmospheric state is required. We have opted to 
use output from a mesoscale model as a surrogate for 
the true atmospheric state in an observing system sim- 
ulation experiment (OSSE). Using a mesoscale model 
in this way has advantages over the analytical functions 
used by previous authors to examine objective analysis 
precision since a full spectrum of realistic meteorolog- 
ical conditions can be considered while the error char- 
acteristics of the observing systems can be simulated. 
By combining realistic atmospheric conditions with 
simulated observations, objective analysis errors can be 
quantified and the statistical precision of the objective 
analysis algorithms can be ascertained. 

a. OSSE Methodology 

Output from the Mesoscale Analysis and Prediction 
System (MAPS; Benjamin 1990) is used. The model 
output is produced on a 60-km polar stereographic grid 
in pressure coordinates. Although wind profiler diag- 
nostics are normally performed in geometric height co- 
ordinates, the vertical coordinate is largely irrelevant 
for the purpose of the experiment performed here since 
intercomparisons between quantities diagnosed from 
simulated observations and the model output are con- 
sidered. Our goal is to create a statistical sample of 
objective analysis errors representative of the full con- 
tinuum of meteorological situations that occur in the 
Kansas -Oklahoma region. Nearly the entire MAPS 
model domain (from approximately 20°N, 120°W in 
the southwest to 55°N, 60°W in the northeast) is, there - 


Table 4. The data polygons used in the observing system 
simulation experiment (OSSE). See Fig. 1 for the geographic 
distribution of the sites. In the OSSE, the L3 or L4 polygon used in 
each case was determined randomly. 


L3 HBR, VCl, HKL, PRC, HVL, NDS 

L4 HBR, VCl, HKL. PRC. HVL, HBR, NDS, PRC 

L5 HVL, HBR, NDS, PRC, VCl 

L6 HVL, HBR, NDS. HKL, PRC, VCl 


fore, considered and the period examined extends over 
many months. 

Starting at the northwest comer of the model grid 
and the lowest vertical level, an array of simulated ob- 
serving points is defined, identical in their spatial dis- 
tribution to the inner array of the WPDN. The data at 
the MAPS grid points within the region bounded by the 
array of simulated observing sites are used to ascertain 
the actual area-averaged diagnostic terms. The area- 
averaged quantities derived from the model gridpoints 
include the horizontal wind components, temperature, 
wind component and temperature gradient terms, the 
horizontal velocity divergence, relative vonicity, tem- 
perature advection, and advective acceleration. These 
area-averaged quantities calculated from the model grid 
points are taken to represent fi or the desired pan of 
the synoptic-scale atmospheric signal. 

Simulated observations of the wind components and 
temperature are determined at the prescribed observing 
sites from the model gridpoint data. Temperature pro- 
files are generated by interpolating the model-predicted 
temperature to the observing site locations. Unlike ra- 
diosonde data, however, wind profiler hourly data do 
not necessarily represent single-time observations but 
are composed generally of temporal means calculated 
from filtered 6-min Doppler moments. Therefore, in 
order to simulate faithfully actual wind profiler obser- 
vations, the model’s wind field is used to reconstruct 
approximately what a profiler would observe at a par- 
ticular vertical level. We construct a wind field stream- 
line that represents 30-min upstream and 30-min down- 
stream advecting distances. The horizontal wind com- 
ponents are sampled at ten points equally spaced in 
time along the streamline. The wind velocity at these 
points is taken to represent the eastward and northward 
moments data. The components are used to create the 
simulated wind profiler hourly values. This technique 
assumes no vertical advection and stationarity in the 
dynamical evolution of the wind field. While this pro- 
cedure does not reproduce actual wind profiler obser- 
vations exactly, it is more realistic than simply taking 
the gridpoint wind values at the observing locations. 
The largest differences between the gridpoint data and 
the simulated profiler data tend to be on the order of 4 
m s _1 and occur near jet entrance and exit regions. The 
mean difference tends to be less than 1 ms’ 1 . 

Following the example of Belt and Fuelberg (1982), 
random error is added to each of the simulated obser- 
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Fig. 6. Cumulative error statistics from the OSSE for (a) wind components, (b) temperature, (c)-(h) the 
indicated derivative-related quantities, with a key showing which objective analysis model is used in the 
comer of each graph. Abscissa values are fractions of the scale magnitudes listed in Table 1. 


vations composing the seven vertical profiles. The ob- 
servational error added to the observations has an rms 
value of 1.5 m s -l for the wind components and 0.5 K 
for the temperature. The perturbed data are then used 
as input to the objective analysis algorithms [Eq. ( 1 )] 
where planar surfaces are fitted to various polygons of 
observing sites ranging from triangles to the full hex- 
agonal network. The polygons used in this exercise are 
listed in Table 4. The objective analysis results and the 
simulated data are differenced from the model’s area- 
mean quantities and the differences stored as histogram 
counts that are a function of rms observational error 
and diagnostic quantity. 

This procedure is followed at each model level. After 
the full vertical column has been considered at a par- 
ticular location, the simulated observational array is 
moved eastward three grid points and the process is 
repeated. After the entire model domain has been ex- 
amined, the next MAPS analysis (typically 3 h later) 


is processed similarly. We have examined model out- 
put from late March 1994 to late October 1994, result- 
ing in approximately 10* separate comparisons for each 
polygon type. While this OSSE is able to simulate 
many aspects of the observational data, certain char- 
acteristics of the data are not accounted for. These in- 
clude the influence of small-scale phenomena on the 
wind and temperature observations. Also, we did not 
attempt to simulate the advection of radiosondes with 
the wind, nor was any attempt made to simulate situ- 
ations where the observational noise was correlated 
across the data network. The results presented below 
are, therefore, slightly biased toward lower error. 

b. OSSE results 

The results of the OSSE are presented in terms of 
cumulative percentile statistics in Fig. 6. The horizontal 
wind components and temperature (Figs. 6a and 6b) 
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Fraction of Scale Fraction of Scale 

Fig. 6. ( Continued ) 


demonstrate no sensitivity to the objective analysis 
model used and can generally be estimated with high 
precision (relative to their scales). The wind compo- 
nents are well resolved (within 25% of their scale 
value) more than 80% of the time and are marginally 
resolved (within 50% of their scale) 98% of the time. 
Objectively analyzed temperature estimates demon- 
strate a somewhat higher degree of precision, being 
well resolved in 96% of cases and marginally resolved 
99% of the time. Experiments were also performed 
with higher levels of rms error in the temperature ob- 
servations (not shown). The motivation for this is the 
installation of temperature profiling remote sensors 
known as radio acoustic sounding systems (RASS) at 
several profiler sites. These instruments provide con- 
tinuous soundings of virtual temperature through the 
lower troposphere but with somewhat less precision 
than radiosonde observations. For rms observational 
uncertainty more typical of RASS (1.5 K), less than 
10% of cases were well resolved and less than 50% of 
cases were interpolated with marginal precision. 


The sensitivity to the objective analysis model be- 
comes more obvious when considering the spatial de- 
rivative terms and quantities derived from them. As 
expected, the degree of precision that can be anticipated 
depends in a straightforward manner on the degree of 
overdetermination of the objective analysis model. Ex- 
act planar fits using data triangles tend to demonstrate 
the minimum precision, while the planar objective 
analysis model utilizing six data points as input tends 
to be most accurate. In general, regardless of the num- 
ber of data points used in the objective analysis model, 
the majority of all cases can be estimated to within a 
particular quantity’s scale value. In other words, the 
correct sign of a particular term can be estimated ac- 
curately in the majority of cases. For quantitative ap- 
plication, however, the fraction of cases demonstrating 
a high degree of precision (within 25% of the scale 
value) is significantly less than found for the wind com- 
ponents and temperature. Consider, for instance, the 
wind gradient components. The exact planar model 
demonstrated precision to within 25% of the scale 
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value only 31% of the time. The fraction increases 
steadily as data points are added to the planar objective 
analysis model with the L6 model being well resolved 
just over one-half of the time. 

Similar conclusions are evident for the remaining 
terms and are summarized in Table 5. In general, the 
temperature gradient terms and the horizontal advective 
tendency of temperature tend to be resolved with high 
precision about one-half of the time. These statistics 
improve somewhat for the L5 and L6 models. Diver- 
gence and vorticity can be estimated with good accu- 
racy about one-third of the time when using triangles 
of observing sites, and, as before, the degree of over- 
determination in the planar surface tends to increase 
the fraction of cases. The improvement is most clearly 
demonstrated in relative vorticity, with the fraction im- 
proving to 57% for the L6 model, while the divergence 
is well resolved less than one-half the time, regardless 
of the model considered. The statistics for the advective 
acceleration are similar, except that even less improve- 
ment is noted in this quantity for the overdetermined 
linear models. The temperature gradient components 
and the associated temperature advection demonstrate 
somewhat higher precision, being well resolved in 
about 45% of cases for triangles and 63% and 55%, 
respectively, for arrays of six temperature observations. 

Objective analysis uncertainty must be considered a 
serious limitation to quantitatively diagnosing field 
characteristics from data using existing observational 
networks. While the spatial objective analysis tech- 
niques considered here are generally capable of deter- 
mining the correct sign of a given quantity, moving 
beyond qualitative applications with any certainty is 
possible in only a surprisingly small fraction of cases. 
The low level of precision demonstrated in the terms 
derived from the spatial derivatives can be understood 
by considering the objective analysis precision statis- 
tics for the horizontal wind components and tempera- 
ture. Even though the level of certainty in the wind 
components and temperature is high relative to their 
scales, the fraction of cases demonstrating precision to 
within 5% of 10% of the scale values are small. This 
indicates that small variations in the wind and temper- 
ature fields that occur naturally are not well character- 
ized in the data. In essence, subtle objective analysis 
errors often dominate the actual physical variability of 
these fields, resulting in imprecise estimates of the hor- 
izontal gradients. 

c. Parameterization of the uncertainty 

The results presented above clearly indicate that the 
wind profiler and radiosonde data streams must be ex- 
amined in close detail in order to identify those situa- 
tions that are amenable to quantitative analysis. Manual 
inspection of the data, however, is not a viable option. 
Not only is a manual filter expensive in terms of time, 
but in many cases even a highly trained meteorologist 


Table 5. The fraction of cases in which the diagnostic quantity 
calculated by the indicated objective analysis model had error 
magnitude equal to or less than one-fourth of the scale value (top 
number) and equal to or less than one-half the scale value (bottom 
number). The quantities listed are the horizontal wind components 
(u. u), temperature (T), wind gradient components (ug), temperature 
gradient components (tg), horizontal divergence (div), relative 
vorticity (vor), horizontal advective acceleration components (ua), 
and horizontal temperature advection (la). 


Model 

uJv 

T 

ug 

tg 

div 

vor 

ua 

u 

L3 

0.82 

0.98 

0.31 

0.44 

0.34 

0.33 

0.32 

0.45 


0.99 

0.99 

0.50 

0.71 

0.50 

0.51 

0.50 

0.68 

14 

0.84 

0.99 

0.40 

0.53 

0.38 

0.42 

0.36 

0.49 


0.99 

0.99 

0.62 

0.82 

0.60 

0.64 

0.57 

0.72 

L5 

0.83 

0.98 

0.44 

0.58 

0.46 

0.48 

0.39 

0.52 


0.99 

0.99 

0.68 

0.84 

0.64 

0.78 

0.62 

0.75 

L6 

0.86 

0.99 

0.52 

0.63 

0.46 

0.57 

0.42 

0.55 


0.99 

0.99 

0.78 

0.88 

0.72 

0.83 

0.64 

0.79 


would be unable to discriminate accurately between the 
subtle variability imposed by synoptic-scale signal and 
that imposed by objective analysis error. Situations 
such as that documented in the case study, as well as 
error caused by highly erroneous observations, are rel- 
atively easy to identify. More typically, however, the 
bias in the analysis results tends to occur over deep 
atmospheric layers and often appears wholly reason- 
able relative to the meteorology of the situation. 

To remove the subjectivity and expense of manual 
filtering, we have attempted to parameterize the objec- 
tive analysis uncertainty. When an objective analysis 
model is applied to an array of observations, estimates 
of the area-mean wind components and temperature 
v es ,, or T c$l ) are returned, as well as estimates of 
the first-order spatial derivatives. Assuming that obser- 
vations of the wind components and temperature (w^, 
Uobsi Tobs) are available near the observational centroid 
(as at the ARM site near Lament, Oklahoma), we at- 
tempt to parameterize the uncertainty in the objectively 
analyzed spatial derivatives in terms of the observed 
difference between the objectively analyzed and ob- 
served wind components and temperature. Hereafter 
we refer to this error in the objectively analyzed wind 
components and temperature as <5 0 (i.e., <5 0 = u ctl - u 0 bs ) 
and the uncertainty in the objectively analyzed spatial 
derivatives as e x . 

The operating hypothesis of this parameterization is 
that a linear relationship exists between the cumulative 
statistics of <5 0 and the cumulative statistics of e x . This 
does not necessarily imply a direct relationship be- 
tween Ci and S 0 on a case by case basis but does imply 
that the fraction of cases exhibiting error of <5 0 is, based 
on that fraction of cases, indicative of an upper limit 
for £ t , €?**. This is illustrated schematically in Fig. 7. 
Consider a particular situation where the difference be- 
tween an observed horizontal wind component and the 
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Fig. 7. Schematic illustration of the parameterization of maximum likely in terms of observed 6 0 . 
An upper limit to Cj is estimated in terms of the observed value of 6 0 . 


objectively analyzed horizontal wind components is 2.2 
m s“ ! . We find from the OSSE cumulative statistics 
that the objectively analyzed wind components are in 
error by 2.2 ms" 1 or less 78% of the time. We assume 
that this cumulative fraction (78%) is applicable to the 
other objective analysis products derived from the par- 
ticular realization. In other words, the spatial gradient 
terms returned from the objective analysis model are 
as accurate as the most accurate 78% of cases as de- 
termined from the OSSE statistics. This equates to an 
uncertainty (c?**) in the horizontal spatial derivatives 
of one-half of the scale value or 0.5 x 10“ 5 s“*. 

To verify the parameterization of e we performed 

an identical OSSE to that described above except that 
Ci was considered in terms of 6 0 assuming an rms error 
of 1.5 ms' 1 in the wind components and 0.5 K in the 
temperature. Results are presented in Figs. 8 and 9. We 
find that beyond some minimum 6 0 , the errors in the 
first-order terms are directly related to the error in the 
associated wind components and temperature, although 
this relationship is more evident for the horizontal wind 
than for the temperature. The relationship also tends to 
be strongest for the L3 model and tends to decrease as 
the degree of overdetermination in the objective anal- 
ysis model increases. 

The parameterization of described above and 
shown schematically in Fig. 7 tends to overpredict the 
uncertainty in the spatial derivatives for 6 0 greater than 
some critical value and underpredict the uncertainty for 
So less than this critical value. We assumed that as 6 0 
approaches zero, e 1“* also approaches zero. This as- 
sumption fails for two reasons. First, owing to trunca- 
tion error, there is some probability that for an incor- 
rectly diagnosed spatial derivative, a correct estimate 
of the wind component or temperature will be returned. 
This effect is compounded by rms error in the obser- 


vations. These random errors tend to cancel for de- 
termination of the zeroth-order terms since the zer- 
oth-order terms are approximately weighted averages 
of the observations. The spatial derivatives, however, 
are influenced by each observation and the error in 
each observation influences the diagnosed spatial de- 
rivatives. These factors limit our ability to parame- 
terize e?** when <5 0 decreases below some critical 
value (<5o ra )* This limit can be seen in Figs. 8 and 9 
where the percentile curves decrease to a minimum 
value and remain constant (e' im ). The e 1 /" 1 is essen- 
tially the maximum precision that can be prescribed 
to a diagnosis of the spatial derivatives. While <5o m 
does not change appreciably for increasingly over- 
determined planar models, e i ira does tend to decrease 
as the number of observations is increased. Not only 
are the overdetermined linear models more accurate 
generally, but our ability to constrain the error bars 
on the resulting analysis product is enhanced. It 
should be noted, however, that e\ ,fn tends to remain a 
significant fraction of the scale values of the quan- 
tities being considered. In other words, it is difficult 
to identify unambiguously those cases where the ob- 
jective analysis results are more accurate than ej m . 
Not only do highly accurate characterizations of the 
spatial derivatives seldom occur, but it is difficult to 
know when they do occur. 

Considering these results, a practical parameteriza- 
tion of e?** can be formulated. As before, we parame- 
terize in terms of 6 0 except that we follow the 90th 
percentile curves shown in Figs. 8 and 9. The equations 
for this parameterization are 

e ™ = Mo + *!*>#■ 

lefU 
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L3 Objective Analysis Model 



Error in Wind Component (5^ 


L4 Objective Analysis Model 



J 0.0 


2.0 4.0 6.0 8.0 

Error in Wind Component (8q) 


10.0 


L5 Objective Analysis Model 




Error in Wind Component 


Fig. 8. Error in the wind gradient components as a function of the error in the wind components assuming 
determined from the observing system simulation experiment (solid lines) and the parameterization (dashed 
lines) illustrated schematically in Fig. 7. An rms error of 1.5 m s’ 1 in the wind components was assumed. 
The solid lines denote cumulative percentiles (percentile values indicated in boxes) and indicate the per- 
centage of cases that had error magnitude equal to or less than the value indicated on the ordinate. The 
ordinate values have units of 10 -3 s*‘ and the abscissa as units of meters per second. 


where e 1 ?** is in units of 10 ~ 5 s" 1 for the wind gradient 
components and 10“ 5 K m -1 for the temperature gradient 
components. Since, as 6 0 becomes greater than <5o im , 
cf** increases nearly linearly with increasing <5 0 , we ap- 
proximate this increase with a straight line. Therefore, m 
is the slope of that line and b is the intercept. The con- 
stants of the parameterization are listed in Table 6. 

Practical application of Eq. (2 ) requires that the cen- 
tral observations of the temperature and horizontal 
wind not be included in the objective analysis scheme. 
However, we consider estimation of uncertainty so crit- 
ical for quantitative application of the results that with- 
holding the central observation for quality control pur- 
poses is necessary. In practice, after an objective anal- 
ysis model has been applied to the available data, the 
interpolated value of u, u, or T is compared to the cen- 
tral observation of w, v, or T to determine the magnitude 
of 6 0 . The uncertainty in the spatial derivative, e™**, 


then follows from application of Eq. (2). The resulting 
value ofcr* can be interpreted as the magnitude of the 
uncertainty in the spatial derivatives at the 90% confi- 
dence level. 

4. Summary and conclusions 

The proposed use of dynamic and thermodynamic 
fields that have been derived diagnostically from wind 
profiler and radiosonde networks by objective analysis 
has led us to consider the accuracy of those fields. Pre- 
vious authors have examined this problem from a theo- 
retical aspect but have not addressed the issue of estab- 
lishing practical estimates of accuracy when consider- 
ing actual observations in an operational mode. The 
uncertainty in the diagnosed values produced by rms 
observational error alone is a significant fraction of the 
scale value (see Tables 1 and 2) of each diagnostic term 
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L3 Objective Analysis Model 



Error in Temperature (g Q ) 



L5 Objective Analysis Model 




Fig. 9. As in Fig. 8 except for the temperature gradient components as a function of error in temperature. 
The ordinate has units of 10~ 5 K m" 1 and the abscissa has units of kelvins. An rms error of 0.5 K in the 
temperature was assumed. 


regardless of the objective analysis model being used. 
However, error introduced by the misspecification of 
the approximating functional model (truncation error) 
can iso produce substantial error. The case study de- 
scribed above shows that the truncation error can se- 
riously bias objective analysis results even in meteo- 
rological situations influenced primarily by synoptic- 
scale features. 

When considering atmospheric data, it is generally 
impossible to separate the influence of observational 
and truncation error in the objective analysis results. 
Given the magnitudes of error that tend to occur in the 
approximation of horizontal derivatives from data, 
some idea of the statistical uncertainty in objective 
analysis results is crucial for further quantitative appli- 
cation of results. The results of the OSSE indicate that 
in nearly all cases, the quantity under consideration 
could be estimated to within the scale value of that 
quantity. However, the results, summarized in the Ta- 
ble 5 also show that well-resolved situations (error less 


than 25% of the scale value) occur in no more than 
one-half of all cases for terms dependent on the diag- 
nosed spatial derivatives. 

Scientific use of the objectively analyzed wind and 
temperature data dictate that the uncertainty in the re- 
sults be known or at least estimated to reasonable ac- 
curacy. Since neither the actual observational error nor 
the truncation error can be known, we derived a param- 
eterization for the uncertainty in the spatial derivatives 
( c ?**) based on the observed error in the objectively 
analyzed wind components and temperature (<5 0 ). The 
eP tends to reach a lower limit (c ? m ) for some partic- 
ular value of 6 0 . This lower limit is the maximum pre- 
cision that can be prescribed to any diagnosis of a first- 
order spatial derivative and depends primarily on the 
rms error in the observations. The e\ m decreases sub- 
stantially (i.e., the precision increases) as the overde- 
termination of the least squares solution is increased. 

Several additional conclusions can be drawn from 
the results presented here. 
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Table 6. Constants of the error parameterization [Eq (2)] 
discussed in the text. For the wind component gradient terms (ug) 
the units of m, b , fjT, 6o m arc 10~ 3 10 -3 s“\ 10" 5 s“ l , and m s~\ 

respectively. For the temperature gradient components (Tg) the units 
of m, b . elr, 6% m are 10‘ 3 m" 1 , 10" 3 K m' 1 . 10‘ 5 K m' 1 , and K, 
respectively. 


Model 


ug 



Tg 


m 

b 

_ Urn 

f 0 

<5 o" 

m 

b 

, Inn 

Co 

61* 

L3 

0.62 

-0.79 

1.30 

3.30 

0.53 

-0.15 

0.44 

0.95 

L4 

0.42 

-0.36 

1. 10 

3.30 

0.18 

-0.31 

0.37 

1.10 

L5 

0.34 

-0.17 

0.97 

3.40 

0.15 

0.25 

0.33 

1.10 

L6 

0.22 

0.02 

0.75 

3.40 

0.12 

0.19 

0.29 

1.00 


1 ) Hourly objective analysis products derived from 
wind profiler polygons of less than the six-station inner 
array and the algorithms considered in this paper 
should be considered of insufficient accuracy to per- 
form quantitative evaluation of kinematic atmospheric 
structure. 

2) Unless careful visual inspection suggests other- 
wise, diagnoses of thermal gradients and advective ten- 
dencies of temperature using data from the five ARM- 
sponsored radiosonde sites around and within the inner- 
array region should also be considered of insufficient 
accuracy to generate boundary conditions for single- 
column modeling applications. 

3) Despite the previous two points, closely moni- 
toring the experimental data stream using, initially, the 
uncertainty parameterization presented earlier and then 
careful scientific evaluation, should allow identification 
of cases that are sufficiently well characterized. It 
should be realized, however, that these cases will be 
limited in number and will likely exclude many typical 
synoptic- scale dynamical regimes, such as jet streak 
entrance or exit regions and frontal zones. 

In the context of the single-column modeling appli- 
cation, our results and conclusions are simultaneously 
encouraging and discouraging. The encouraging aspect 
is that there are a wide variety of situations in which 
the diagnosed fields are of sufficient accuracy to permit 
their use as forcing functions for SCMs. Furthermore, 
we have suggested a simple, and somewhat crude, pa- 
rameterization that can be used to decide when the di- 
agnosed fields can be used. The discouraging aspect is 
that the fields are most likely to be inaccurate in situ- 
ations where the actual fields have strong spatial and 
temporal gradients. These often will be the very con- 
ditions in which one would like to use the diagnosed 
fields. An alternative to using the objectively analyzed 
fields is to use fields produced by data assimilation 
models. In some sense, data assimilation model fields 
may be seen as objectively analyzed or interpolated by 
the model itself, but in a way that produces balanced 
and consistent results. Considerable research remains 
to be done using both objectively analyzed fields and 


data assimilation output in order to evaluate the use- 
fulness and accuracy of these two approaches when 
applied in an operational mode. 
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